THRESHOLD ESTIMATION IN AUTOREGRESSIVE MODELS 
DRIVEN BY COLORED NOISE 
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Abstract. Large sample statistical analysis of threshold autoregressive (TAR) models 
is usually based on the assumption that the underlying driving noise is white. In this 
paper, we consider a model, driven by Gaussian colored noise with geometric correlation 
tail and derive a complete characterization of the asymptotic distribution for the Bayes 
estimator of the threshold parameter. 



1. Introduction: the setting and the main result 
Let (Xj)j & z + be the sequence generated by the recursion 

X j = (p+Hxj-x^e} + P-^-{X j - 1 <9})X j ^ 1 + ej, j > 1 (1.1) 

where (ej)jgz+ is a random process with known distribution, p + and p_ are known 
constants and 9 is the unknown threshold parameter to be estimated from the sample 
X n := (Xi, ...,X n ). The equation (jl.ip is a basic instance of the threshold autoregression 
(TAR) models, which play a considerable role in the theory and practice of time series. 
This type of models have been studied by statisticians for already more than three decades, 
producing interesting theory and finding many important applications, some of which can 
be traced in the early and more recent surveys [13] and [TJ], [TE], [5], [3]. 

When it comes to the asymptotic analysis of the estimators, the standard conditions 
imposed on the models such as (jl.ip is strong ergodicity of the observed process (Xj) and 
independence of e^'s. Departure from these assumptions often poses challenging problems. 
While the former assumption has been relaxed by a number of authors [13], [T], [17], the 
independence assumption of the driving noise sequence has not yet been addressed. As we 
shall see shortly, in the dependent case the problem falls into the framework of statistical 
inference of hidden Markov models (HMM), where the driving noise plays the role of the 
hidden signal (see Ch. 10-12 in [2] and the references therein). However, most of the HMM 
literature deals with locally asymptotically normal (LAN) experiments and, to the best 
of our knowledge, non-LAN models with partial observations have not yet been studied 
systematically. 

In this paper, we consider the model (jl.ip in which (e^) is a sequence with geometrically 
decaying correlation. More precisely, let X = (Xj)j £ z + be generated by the recursion 
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subject to Xq ~ N(0,1), where p := |/?+| V \p-\ < 1 and the unknown parameter 
takes values in an open bounded subset C R. We shall consider the problem with 
discontinuous drift function f(x, 9) := (p + l{ x> gy + p-l{ x< g})x, and thus assume p+ ^ p- 
and 0. The driving noises (£j)j<=z + and (Cj)jez + are independent: the white noise 
component (Ej) is a sequence of i.i.d. iV(0, 1) random variables and the colored noise (£j) 
is the Gaussian process, generated by the linear recursion 

& = a&-i + Ci, i>!> (1-3) 

where (Cj).?€Z + are i.i.d. N(0, 1) random variables and a is a known constant |a| < 1, 
controlling the bandwidth of the noise. As we shall see below, the threshold 6 can be 
estimated at the rate of n and hence the asymptotic analysis of the estimators for the 
other parameters, such as p+, /)_ and a, can be essentially carried out within the LAN 
framework (see [3]). 

All the aforementioned random variables are defined on a measurable space (0,3"), 
with the family of probabilities (Pe) eee , indexed by the unknown parameter. For integers 
k > m, we define 3" fc ,m := o-{d,Ei k < i < m} and set 3~ m := 3~ ,m and J kjOQ := Vi>fc ^k,i- 
All the processes in our problem are adapted to the filtration (3 r j)jez+ and we shall assume 
that 3" = 3~o,oo- Finally we define the observed filtration 3^ := cr{X;, i < j} C 3j. 

The recursions (II. 2\ and (11. 3p form a conditionally Gaussian system, which means that 
the conditional law of £ n given X n is Gaussian, and by Theorem 13.5 in [10] 



X 3 = fiXj-uO) + 0-1 (0) + V! + 7i-i% (1-4) 
where (e})jgz + is the innovation sequence of i.i.d. A r (0, 1) random variables. The process 

:= ^sfe'l'^f ) an d the deterministic sequence 7j := Ee(£j(0)— £j) 2 satisfy the Kalman 
filter equations 

§(0) = o§_i(0) + T^^fc - f(Xj^e) - §■_!((?)) (1.5) 
1 + 7j-i v y 

2 2 

7, = «Vi + 1 - A^", (1-6) 
1 + 7i-i 

subject to = and 70 = var(£o)- 

To avoid inessential technicalities, we shall assume that £0 ~ ^(0,7), where 7 is the 
unique positive root of the equation 

2 ■ 1 ° 2 7 2 
7 = a 7 + 1 



1 + 7 

In this case, the conditional mean £•,(#) satisfies (|1.5p with constant coefficients: 

%(6) = ag-iW + ^(Xj ~ /(*;-i, 0) " S-iW). (1-7) 

It can be seen that (7^) converges to 7 exponentially fast and all the results claimed below 
hold for £0 with an arbitrary Gaussian distribution. 

Let X Q := X and Xj := Xj - f(Xj-i,9), j > 1 and note that 3^ 1 = $f_ v By the 
definition of the conditional expectation, — ij-i is orthogonal to Moreover, 



THRESHOLD ESTIMATION IN MODELS WITH COLORED NOISE 



3 



since the process (Xj,£j,£j) is Gaussian, — is independent of 3*f-\ and thus 
of $f_i as well. Further, since t/1 + 76) = — + and is independent of 
3^-1, independence of ej and 3"^i follows and the innovation form (II. 4p implies that the 
likelihood of the data X n is given by 

L„(X";9) = -^exp(-ix 2 )x 

(TITO) " exp ("51^ % ^ ~ / w - " " S - w ) 2 ) (L8) 

The likelihood function is discontinuous in and hence we are faced with a singular statis- 
tical experiment. In such problems, the maximum likelihood estimator is often asymptot- 
ically inferior to the Bayes estimator 9 n and the latter is typically asymptotically efficient 
for arbitrary positive priors in the following minimax sense (see Theorem 9.1, [9]): 

lim lim inf sup Kg(T n — #) 2 > lim lim sup E#(0 n — #) 2 , 
<5^0„^oo T„ 0.|0_0 O |<<5 <5->0 n -too 6:\6-G \<5 

where T n 's are 3"^-measurable statistics. 

Another advantage of the Bayes estimator for the problem at hand is its relatively low 
computational complexity. Note that the likelihood function is piecewise constant in 9 and 
has at most n jumps at {Xq, ...,X n -\}. Hence for a prior density tt, the Bayes estimator 
with respect to the quadratic risk is given by 

- f e eL„{x n ;0)A«)<i« _ IWxu-.i.x,,,}!-^ M*"; fa) Sx"'. n e < e )< u 

where X(~-\ is the j-th order statistic of X n . If the prior tt is chosen so that numerical 

integration in the right hand side is avoided, the computation of 9 n can be carried out in 
polynomial time of order 0(n 2 ). 

The main result of this paper is the following characterization of the asymptotic distri- 
bution of the sequence of Bayes estimators: 

Theorem 1.1. Let (9 n ) be the sequence of the Bayes estimators with respect to the qua- 
dratic loss function and a prior with continuous positive density tt. Then for any contin- 
uous function <p with at most polynomial growth 

limE fl(J (n(0 n - O )) = ^ej(u), 

uniformly on compacts from 0, where 

f R Z(u)du 
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and In Z(u), u 6M is the following two sided compound Poisson process: 
\nZ{u) = 



E?= + i W ( fief - \f? ) ->0 



(1.9) 



_ E;=r lj (Pej - ¥ 2 ) u < o 

Here Tl + , Ii~ are i.i.d Poisson processes with the intensity 

to = / p(9o,y,9 )dy, 



p(x,y;9o) is the unique invariant probability density of the Markov process (Xj,£j) under 
¥g , (e^) are i.i.d. N(0,1) random variables, independent o/II + and H~ and 

p = \ vr+7 ) (Wir^gliT^ J=^ + -p-) (1+7)(1+72) - 

The proof of this theorem is given in the next section and some supplementary results, 
concerning the ergodic properties of the relevant processes, appear in Appendix 1X1 

2. The proof 

2.1. Some notations. The actual unknown value of the parameter will be denoted by 9q 
and will be assumed to belong to a generic compact K C 0. We shall use Cj, i £ N to de- 
note absolute constants, whose values depend only on the known parameters of the model 
and the compact K and may change at each appearance. For random sequences (x n ), (y n ) 
and a positive real decreasing sequence (r n ), x n = y n + 0{r n ) means that sup n \x n — y n \/r n 
is a random variable with moments, bounded uniformly over K. Throughout, we reserve 

a a a ^ 

and c := 



1+7 1+7 
For an integer n, the quantities such as n 1 / 2 and n 1//4 , are understood to be rounded to the 
nearest integer. For I < k, we set Ylj=k(--) = an< ^ Ilj=fc(— ) = 1- For a vector z € R d , 
II^H stands for the ^i-norm. Finally, Pg and E# denote the probability on (0,3") and 
the corresponding expectation, under which all the processes are stationary (the unique 
existence of such probability is argued in Appendix lAl) . 

2.2. Preliminaries. Consider the scaled sequence of likelihoods 

L n (X n ;9 + u/n) 
Zn(u) := , r , u £ U n := n(9 - (9 ), n > 1. 

-^n(^ ; foj 
The Bayes estimator of 9 is given by 

~ _ f @ 9L n (X n ;9)-Tr(9)d9 _ f Vn (9 + u/n)L n (X n ; 9 + u/n)vr(6lo + u/n)du _ 
° n = f e L n (X n ;9)n(9)d9 = Jj~ L n (X«; O + u/n)7r(0 o + u/n)du = 

1 L uL„(X n \ 9 n + u/n)-K(9n + u/n)du 
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and thus 

~ _ /u„ uZ n (u)ir(6 + u/n)du 

° J Vn Z n (u)ir(6 + u/n)du ' 

The right hand side is an integral functional of Z n (u), u £ U n , which under appropriate 
tightness conditions, converges weakly to the random variable 

_ _ J R uZ n (u)du 
J R Z n (u)du ' 

if the finite dimensional distributions of Z n (u) converge to those of Z{u). More precisely, 
the result claimed in Theorem 11.11 follows from Theorem 1.10.2 in [9], whose assumptions 
we check in subsection 12.31 and 12.41 below. 



2.3. Convergence of finite dimensional distributions. We shall prove that the char- 
acteristic functions of the finite dimensional distributions of log-likelihoods Y n (u) := 
In Z n (u) converge to those of the compound Poisson process, described in (|1.9p . To this 
end, we will show that for any d > 1 and real numbers U-d < ... < 



ip n (X) := E eo exp I ^ iX k (Y n (u k ) - Y n (u k -if) ) 
\k=-d ) 



n—toc 
> 



f[ exp[ ro (^-^_ 1 )(e-^^ 2 -^ 2A i-l)) =:V(A), VA G R d , (2.1) 
k=-d ^ ' 

uniformly over compacts from 0, where w and (5 are constants, defined in Theorem 11.11 
Without loss of generality, we shall assume that no = and consider only positive life's. 
The symmetric case of negative u k s is treated similarly and independence of the emerging 
compound Poisson processes Il + and LT~ will be clear from the proof. 
Let Hj be the vector with the entries 

Sj:=£-(0 o + Ufc/n), k = 0,...,d. (2.2) 



Using the likelihood expression (jl.8p . we get 

Y n (uk)-Y n (uk-i) = 

1 3=1 

7 3=1 
71 1 1 

3=1 " 

(2Xj - f(X^ h 9 + u k /n) - f(Xj_ u 9 + « fc _i/n) - Ej_ a - Sjlj] 



(2.3) 
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where we set '■- 



and used the identity 



f(x, 9 + Uk-i/n) - f(x, 6 + u k /n) = x{p + - p^)l {x€D k } . 



If we define 



5«,- 



;fe-l 



and a k - 1 



the expression (|2.3p takes the following form under Pg 

n j 



with 



'0; #0 + 



^(x j - 1 (p+-p-)i {x ._ ieDi , } + 8];_ 1 

. The sequences (<$) and (c^ -1 ) satisfy the recursions 



(2.4) 



n 

k _uxk 



5 i = bS j-l ~ <P+ ~ / 9 -) X i-l 1 {X 3 _ l6 Dfe}, 3 > 1 

So =0, 



and 



fe-i 



c(p+ - p_)X,-_il {x ._ i6B *-i } , i > 1 



<7, 



1+7 



=0 



c := 



(2.5) 



(2.6) 



07 
1+7 



. In what follows, both representations (|2.3|) and (|2. 



where we set b :- 
will be useful. 

To prove the convergence (|2.ip , we shall partition the terms in the sum (|2.3p or (|2.4p into 
n 1 / 2 consecutive blocks of size n 1 / 2 and discard n 1 / 4 first terms in each block. As shown 
in the Lemma 12. II below, discarding the total of re 1 / 4 • n 1//2 terms does not alter the limit 
of the sum and, by Lemma 12.21 the remaining blocks become approximately independent 
due to the fast mixing of the process (Xj, Sy). Moreover, in each remained block, the 
probability of having exactly one of the events {Xj-i 6 D^} occurred is of order n 1 / 2 . 
Hence the sum of n 1 / 2 such nearly independent blocks yields the compound Poisson limit 
of Lemma 12.31 This approach to Poisson limits dates back to at least . 

Denote by Sj t k the summands in the right hand side of (I2.3P or (I2.4p . Set 



and, for m = 1, ...,n 1//2 , define 



Sn '■— ^ ] Aft ^ ^ 

fe=l j=l 



<i ran 1 / 2 

k=l j = ( m _l) n l/2 +n l/4 
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By the triangle inequality 

|Vn(A)-^(A)| <Eg 



exp (iS n ) - exp ( i ^ 

^ m=l ' 

E 9o exp I i ^ S n,m J - ( Eg exp (iS n> i) J 

\ nl/2 

E^o exp (i5n,i) ) - V'(A) 



+ 



(2.7) 



where Eg stands for the expectation with respect to the probability ¥g on (fi, jF), under 
which the process (Xj Ej ) is stationary (see Lemma lA.3p . In the following lemmas we 
show that all three terms in the right hand side of ()2.7[) vanish as n — > oo, uniformly over 
#0 on compacts from 0. 



Lemma 2.1. For any A 6 K and a compact K C 0, 



lim sup Eg, 



Sri ^ ] ^n,' 



m=l 



and consequently 



limE, 



0o 



exp (iS n ) — exp 



6eO 

^ m=l ' 



0, 



(2. 



(2.9) 



uniformly over 9q £ K. 



Proof. We shall assume that n is large enough so that max^. \ 9q + Uf./n\ < sup \K | + 1 and 
hence on the events {Xj-± G D%} and G -B^}, we have < sup \K\ + 1. Using 

the representation (|2.4|) . we get 



E So 



.1/2 



m=l 



< 



n 3/4 d max |A fe | maxE eo X i _i(p+ - p-)l{ Xj - 1 eDk} + Oj-i 



(2.10) 



V 1 + 1^3 + " P-)l {x ._ ieB k-i } + — i +-(x j „ 1 (p+ - p-)l {Xj _ ie D^} + $-1 

By Jensen's inequality, it follows from (|2.5|) . that 

E 0O (^) 2 < IftlEfl,,^) 2 + Y^c\p + - p-) 2 (sup\K\ + lfFg^Xj., G !>*), 



which, in view of (jA.ip and <5q = 0, implies maxj< n E eo (<5 fc ) < d/n. Further, since e 



is independent of 3yLi, E0 o |5jLi||e)| = IE 0O |M(9 |gj | < C 2 /n. Similarly E0 o (<r* x ) 2 < 
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Czjn. Plugging these bounds into (|2.10p . we obtain (12.8j) 

n 1 / 2 



■m=l 



with a constant C4, depending only on K. The uniform convergence in (|2.9p follows, since 
I exp(ix) — exp(iy)| < \x — y\. □ 



Lemma 2.2. For any A € 



lim 



( nl/2 \ /- \ nl/2 

K d(j exp ( i ^2 S n , m J - ( E 0O exp (iS n ,i) J 



0. 



uniformly over 9q £ if /or any compact K C 0. 



Proof. We shall use the bound (|A,7|) of Lemma IA.3I and thus will need to establish the 
corresponding Lipschitz property. To this end, for fixed x,y £ M and z £ let 
(X j (x),^ j (y),E j (z)) be the solution of the recursions (USD, {E3D and (cf. and (ETH ) 



3j = 6Ej_i + c(Xj - O + -u fc /n)j , fc = 0, d, 

subject to the initial conditions x, y and z respectively. The latter recursions give 

j 

E^z) = Vz k + cJ2 fa ~ f(Xi-i, 9 + u k /n))V~\ 
i=i 

Consider the random variable (cf. the right hand side of (|2.3p ) 

d 



(2.11) 



fe=l i=l ^ r 

8fc / \ Sfe-i 



2^- - /(X,_!, 9 + u fc /n) - /(Xj^Oo + uh-x/u) - E^z) - E^(z)) , 

where we dropped the dependence on x and y for brevity. Define the function h(x, y, z) := 
Efl exp (i&£(x, y, z)) . We aim to show that for z, z' £ 

\h(x,y,z) - h(x,y,z')\ < L(l + \x\ + \y\ + \\z\\ + \\z'\\j \\z - z'\\ (2.12) 

for some constant L, independent of t. Using the definition of &e(x,y,z) and the explicit 
formula (I2.1ip . a tedious but straightforward calculation gives 

\& t (x,y,z) -<f> t {x,y,z')\ < 

d / £ 

£ |a*| (\z k - 41 + k-i - 4-il) E 21^(1^1 + 



fc=i 



W + -2 



'iDirifti + 2 E i 6 i j E (1^1 + + I 6 ''"' 
1 1 7=1 i=i / 
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Taking the expectation of both sides, we get 



\h(x,y,z) - h(x,y,z')\ < E<? \^(x, y, z) - $t(x,y,z')\ < \\ z ~ z '\\ x 

£2|&P2E fl0 |X,-| + (||z|| + II^H) +4^|6pXl fe r^o(l^-i| + 16-1 

V j=l j=l i=l 

and applying the bound (|A.2[) . the inequality f|2. 12|) follows. 

Note that by the Markov property, Eg e = Ee h(Xo, £o> Hq), and for m = 1, .... 



n 



1/2 



E ftl (e iS "< m |3~, 



(m-l)n 1 /2 



Ee I /i( ^( m -l)ni/2+nV 4 -l' ^(m-l)n 1 /2 +n l/4_ 1 , S( m _ 1 )„l/2 +n i/4_ 1 J |3"( m _i) n i/2 



Hence by the Lemma lA. 31 



^o(^ m |^( m -i )n i/ 2 )-E eoe 



i,S'„ 



with a positive constant q < 1 and Ci, independent of Oq. Finally, considering the tele- 
scopic series, we get 



E 0Q exp I i ^ S n, 

^ m=l 

n 1 /2_i „i/2_fc „V2 n 1 /2„ fc _i n i/2 

XT ( ^ s 

k=0 ^ 771=1 
nV2_l 



E 6o exp (LSW,i) 



=n 1 /2_fc + i 



m=l 



n e 6 

=n 1 /2-fc 



fc=0 

as claimed 



^ E eo E fla (e iS ».» 1 / a - fc |y (nVa _ fc _ i)nl/2 ) -E^ 1 



< Cm 1 / 2 /' 4 ^> 0, 



Lemma 2.3. For any A G M d ; 



< 



□ 



lim 



E 0O exp (iS n ,i) - V'(A) 



,1/2 



uniformly over 9q € K for any compact K C 0. 
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Proof. Let D n := Ufc=i an d define the events 

re l/2 

Aq = O {Xj^i D n } (all samples avoid D n ) 

A j)k = {X^x G n p| D n ) (only (j - l)-th sample falls in D k n ) 

i^j, i=l 

d nV2 

= Aj^ (a single sample falls in D n ) 

k=l j=l 

A2+ = (Aq U Ai) c (two or more samples fall in D n ), 

and note that 

d n 1 / 2 

E, e i5 -i = E 9o e iS ^l {Ao} + ]T ^ E eo e i5 '^ l { ^ fe} + E 9o e iS ^l {A2+} . (2.13) 

fc=i j=i 



Below we shall show that 

nV2 



^E^e^l^ = e -*^M^2( Ufc - Ufc-Oom-Va + 0(n~ 3 / 4 ) (2.14) 



E^e 15 "- 1 !^} = 1 - Y,( u k ~ u k ^)wn~ 1 / 2 + 0(n~ 3 / 4 ) (2.15) 



fc=i 



P e0 (^ 2+ ) = O(n- 3 / 4 ), (2.16) 

where w := J p(6o,y;6Q)dy and p(x,y;0o) is the unique invariant density of the chain 
(X,-,£j) (see Lemma [A. 2\i . Plugging these expressions into (j2. 13|) . we obtain 

(E^e 15 "- 1 ) 

/ d d v il/2 

1 - J> fc - U^WU^ + £ e -|A fe /3 2 -i/3 2 A 2 |- Ufc _ Ufe _ l)ron -l/2 + ( n -3/ 4) 
V k=l k=l J 

^ exp (j2 {e-^ 2 -^ - l)K - • 

The claimed result follows, once we check that fj2 .151) . (12. and (I2.16P hold uniformly in 
#0 on compacts from G. 

To this end, note that on the event Aj t k, the equations (12.5P give 



51 = 5$, t + k 



and 

srfc _ rkui 



Si = 5« Q b l - c(p + - p^Xj^l^. ^kyV l l {i > j} , 
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where 6q, £ = 1, k are bounded random variables under P# . Similarly, since D% C 1 
for £ > k, and P 6o (Xj-i G B l ~ x n D*) = for £ < jfc, 



-i e-iui 



4~'b\ £€{l,...,k} 



and 



-i e-ivi 



a* l b l - c{ P+ - p_)X i _il {x ._ ieD fc } & 3 l l {i > j} , £€{k + l, d}. 



Hence, for n 1 / 4 <j< n 1 / 2 on the event Aj^, by (j2.4|) we have 

Sn,l=E E = Xk E S ',fc + ( fon ) = 



=1 i=nV4 
nV2 



i=j 

v/iT^e, + - p-)l {Xi _ ieD k } + Sti)) + o(b T 



-A fc 

a, E 



P+ - P~ y ?A. X (P 



2 V yrTr 



1/2 



i=j+l 
,1/2 



i Z' c (p+ - p-) 



Xj^V-^ 1 ) \+o{b nl,i ) 



i=7 



where we defined the kernel 



p- 



i < 
i = 



c(A+-P-),j-i , >0 
I v / T+7 



By the triangle inequality 



< 



E do e iX ^l {A . k} - E eo e iX ^>n {X] _ iem 



+ 



(2.17) 



+ o{b nl/i ) 
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By (COj) . for i < j 



g (Xi G D n ,Xj G L>£) = E eo l {x . eDn} 



U fe - U k -i; 



n 



{X t g D n ) < Cm- 2 , 



with a constant Ci, independent of 9$. Similar bound holds for j < i and hence, using the 
identity 1 {A} - l {AnB} = l{A\(AnB)} = 1 {AnB-}, w e get 



< 

1/2 



00 



ft-ie^n |J {Xi-ieA»})< 



i^j, i=n 1 / 4 



,1/2 



E P eo G D n , G D*) < Cm" 3 / 2 . (2.18) 

i^j, i=n 1 / 4 



Further, since (e^) are i.i.d iV(0, 1) and e$ is independent of for i > j 
Khx^Dl] exp ( - E ~ 3) ~ l 2 XkX ^ E ^ " J')) = 

i=j i=j 
~ /I " 1/2 -i , " 1/2 -i \ 

%l { ^._ ie ^ } exp(--A^ 2 E Q'W-^o E Q 2 « +0(n" 2 ) = 

^ i=0 i=0 ' 

E 0o l { ^._ ie ^} exp - -A 2 /? 2 - i-A fc /3 2 + -9l E Q^X** + iA *0 + °( n " 2 



i=n 1 / 2 -j+i 



By (HI), 



6XP ( ~~ ~ ' 1 'o Xk ^ 2 ) w ^ Uk ~ Uk ~ l ) n 1 +0(n 2 ), 
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< 



ex p(^o E Q 2 «(A^ + iA fc )) -1 

V j =n l/2_j + l ' 



+ 0(n~ 2 ) < 
+ 0(n~ 2 ) < 



(2.19) 



Plugging (f_7[8l) and (i!_T9l) into (|_Tfl) . we obtain 



and in turn (|2.14p : 

n l/2 

1 \ . «2_ I «2 



< C 3 n- 1 6 ,lV2 - J " +C 4 n- 3 / 2 , 



nV2 



< 



+ 



1/4 



E ^ 



+ 



^2 e~^ Xk ^~^ 2x k(u k - Uk-^wn" 1 



< 



,1/2 



Can" 1 E + Cm 1/2 n-^ 2 + ^^0^1^ + 0^ <C,n~^. 

By setting all A^'s to zero, we also get 

d n 1 / 2 d 
F 6o {Ai) = E E V*>( A 3,h) = E( Ufc " ^-i)^" 1/2 + 0(n" 3/4 ). (2.20) 

k=l j=l k=l 

Further, 

= l-^o I U {X,-ie_J n }l > 

d nVa d n i/2 

1 - E E p e (^-i e = i - E E(^ - + 0{n-*/ 2 ) = 

k=l j=l k=l j=l 

d 

1 - E( Ufc " "fc-i)^" 1/2 + 0(™ -3/2 ) (2.21) 
fc=l 
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where in the equality f we used (|A.3h . On the other hand, Fg (Ao) < 1 — Fg (Ai) and the 
estimate (12.151) follows from (I2.20p and (I2.21|) and the asymptotic 

E do e iS ^l {Ao} =F do (Ao) + 0(b nl/4 ). 

Finally, (|2~TfTjl follows since E 0Q (A 2+ ) = 1 - E 0O (A O ) - P^(Ai) = 0(n~ 3 / 4 ). □ 

2.4. Tightness. In this section we check the tightness conditions of Theorem 1.10.2, [9]. 

Lemma 2.4 (condition 1.1 of Theorem 1.10.2, [9]). For any compact K C O, i/iere is a 
constant C > 0, suc/i i/toi 

sup |«a - uil -1 !^ (\/Z n (u2) - V^nK)) 2 < C(l + i? 2 ) , (2.22) 

|ui|<R,|u 2 |<-R v / \ / 

for all 6» G K and R> 0. 

Proof. Suppose U2 > tti, then using the elementary inequality In - > 2(1 — y/x), x > we 
get 



Z n (ui) 



1 



9F M l <r tp i, z »(m) 



Similarly to (|2.3p . we find that under Eg 0+Ul / n , 
ln Z n ( Ul ) _ 1 



z n (u 2 ) T+j g " '-^W-ienW + x 

f\A + 7ej + i(x,-i(p+ - P-)l{x j _ 1 eDO + <f?_i) J, 

where (<5j) is defined in (|2.5p . Note that £j is independent of under P0 o + ui /n an d, as 
in the proof of Lemma [2TTT E, 0o+Ul / n (5j) 2 < C\(u 2 — u\)n~ l . Hence 

Eg (y Z n (u 2 ) - \JZ n {ui)\ < 

n 2 
S^^e Q + Ul /n (Xj-l{p+ - P-) 1 {X J _ 1 GD2} + <5]-l) < C 2 \U 2 ~ Ui|, 

i=i 

where C 2 depends only on K. By symmetry, the same inequality holds when u 2 < iti and 
(I2T221) follows. □ 

Lemma 2.5 (condition 1.2 of Theorem 1.10.2 [9]). For any p > 1, i/iere is a constant 
C(p), such that 

E 6o Z^ 2 (u)<^-, ueU n :=n(@-e ). 
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Proof. The proof is an adaptation of the analogous Lemma 2.2 in [5]. We shall assume 
9q > and u > 0, omitting the similar complementary cases (recall that 0). Note 
that for a constant c > 0, 

EeX/ 2 (u) = ^l\u)l {zlJ2{u)> _ e _ m} +^X /2 (u)l {zl/ , {u)<e _ cu} < 

(p eo (lnZy 2 (u) > -cu)) +e-i", 
and hence it is enough to check the large deviation bound 

F eo (±lnZ n (u) > -cu) < (2.23) 

for some positive constants c and C(p) and all p > 1. 
For «i := u > the formula (|2.4p gives 



1 n 

hxZ n {u) = -— ^ - P-)l{x,_ lG Di} x 

(1 \ " 1 

7 i=i 

where D\ = [#o, 6q + u/n], the sequence (<5j) is generated by (|2.5p with fc = 1, and 

Further, 

P eo (-lnZ n (u) > -cu) = 

^ t(^-.)-5(i"H)>i^H2-)s 



•J 

and since 



».(e(^- 



2 V2 



Vj-i ) ) > cu ) < 



2V2 J 
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the bound (|2,23p holds, if we show that for some positive constant c and all p > 1, 

P*(£»?<«)<^. (2.24) 
v j=o y 

To this end, we shall split the consideration to the cases u < n 3 and u > n 3 , where 
s £ (0, 1) is a constant to be chosen later on, depending on p in (|2.24p . 

Case u < n 3 . In this case, D\ C [# , Oq + ~r=r] an d since 

£j = 6<5j_i - c(p+ - p_)Xj_il { ^_ ieD i } , j > 1, 
subject to 5o = 0, it follows that 

1 



\5j\ < \c\\p + - p-\ [9 + n 



,s-l 



1-|6|- 

'/>+ - p-Kdo + n 3 - 1 ) < \a\\p+ - p-KOo + n 3 - 1 



1 — \a + 7 



where we used the definitions c := and & := and the assumption |a| < 1. Further, 

r2 1 /__. _\2 



1 / \2 

*3 (/>+ -p-) + 8j) 1 {X 3 &Dl} > 



1 + 7 
1 + 7 



h- \a\{9 + n 3 A ) l{A je Di} > 



4W " l a U IfteDi) •= ^ll^GDi} 

where the latter inequality holds for all n large enough. Consequently, 

• n— 1 \ /n— 1 N / n— 1 



V j=0 7 V i=0 J V j=ni/2 

By Lemma |A.2| the process (Xj) is geometric mixing and we have 

r 

(X j eD 1 n )=E 6o / 



i-00+u/n y 1 2 

" So |A 3 fc w n) = ^ / e - 5 (*-&-i-/(^-i.eo)) dt > 



-1e Jo inf e -|K-i-/(^-i^o)) 
n V27T te[e ,e +i] n 

where the constant C2 > can be chosen for all 9o € K to be independent of j and n by 
the ergodic properties of (Xj,£j) from Lemma lA.21 Hence with c := \C\C21 for an integer 
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IT 



P>1, 



0o 



n— 1 \ / n— 1 \ 

E ^ ^ )- 

3=0 J V j=„i/2 7 



n-1 



j=n 



1/2 



> gC 2 « ) < 



n-1 



2p 



1/2 



where we defined := l^.goi} — Ke l^ X:j eDl}- Since 1 77^ ] < 2, by Lemma \A.2\ 



/ n— 1 % 2p / n— 1 >. 2p 

E - E 4 E 



]=n 



1/2 



j=n 



1/2 



< 



n 



2p E fl 



1 /o / / \ . 1 /o / 



n-1 v 2p 



j=n 1 / 

for all n large enough. Hence it is enough to check 

~ I- 1 Y p 



(2.25) 



1/2 



To estimate the latter expectation, we shall apply the covariance inequality (8.1) from [6]: 



n-1 



2a< 



®e E ^ 



E, 



ft, 



1/2 



n— n 1 / 2 — 1 

E 

3=0 



2p 



< 



Apn^ (^e \vo^e (Vj\%)\ 
3=0 



i/p 



(2.26) 



Since \rjo\ < 2, 



Ee a \vo^e (Vj\?o)\ P < ± p ^e \^e (Vj\ ^o)| P < 4? y-C 4 Cr j 



(2.27) 



where the latter inequality holds by Lemma fA.21 since E^^lCTo) = Eg (Eg (r/j|3"j_i)|3"o) 
and lEflo^jl^-i)! < C 4 ^. Plugging the bound (12371) into ([236]) . we obtain (12351) and 
consequently (|2.24p for u <n s . 
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Case u > n s . For a fixed integer k and all j > fc, define I^-i^ := Cfi=j—k^ i & ®l an d 
note that on Tj—i & we have 

jr'-fc 



\Sj\/\p+ -P- 



i=i 



< |c| sup |0| 



|6|' 



1-|6| 



=: C x b k . 



Now let fc' be such that, C\b k ' < \0q, then (recall that both 9q an d u are positive) 



1 + 7 4 

Define Wj := l{x,-eDi}l{r lfe /} an d n °t e that 
E* ^ =E flo l {r ._ lfc} P flo (X J - e ^IX^!,^-!) > 



1 - - -i^-i-/^-!;^)) 1 



inf e 



>C 3 - 



'27r*e© n 
where the positive constant C3 can be chosen independent of j and n due to the ergodic 
properties of (Xj,^j) from Lemma I A. 21 Hence with c := and any integer p > 1, 



^ 3=0 ^j=k' ' 

, n-1 1 v , n-1 v 

*> I E ( W J " E *o^) > 3«0, < (3/C 3 ) 2p+2 ^K d0 ( W i ~ E o Wj) 

^ j=k' ' ^j=k> ' 



2p+2 



for all sufficiently large n. Now (j2.24p follows if we show that for a positive constant C(p), 

/ n-l \ 2p+2 

E e ( E " Ee o W i) ) < C(pK+ 2 , Von s (2.28) 

To this end, we shall use the Marcinkiewicz-Zygmund inequality from [7j. For a sequence 
of random variables (jjj)j^, the coefficient of weak dependence is defined 

C T)q := SUp I COV (Vtx-Vtm^tm+I-Vt^l 

where the supremum is taken over all {ti,...,t q } such that 1 < ti < ... < t q and m, t 
satisfy t m+ \ -t m = t. 

Theorem 2.6 (Theorem 1, (7j). Let {r]j)j^ be a sequence of central random variables 
such that for a fixed integer q > 2, 

C t ^ = 0{t~ q / 2 ) as t^oo. 

Then there exists a positive constant B, independent of n, for which 

■n—X \ q 



(2.29) 



E (E^') 

v 3=0 J 



< Bn q l 2 . 



(2.30) 
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We shall apply this theorem to the bounded sequence rjj := Wj — Kg Wj. Since rjj is a 
function of (Xj-k', ...,Xj), it inherits the mixing property (1A.6P - More precisely, with 

h(x,y) := E 0o (r] tl ...r] tm \X tm = x,£ tm = y), 

g(x,y) := E 0o (rj tm+1 ...r) tq \X tm+1 _ k/ = x,^t m+1 -k' = y) 

by the Markov property of (Xj,£j), for t > k' 



a 



t,q 



^eoVh-Vt, ~ ®e Vh-Vt m ^8 Vt m +i-Vt q 



< 



^e H x tm > it m )g{X tm+1 -k' , 6 ro+ i -v ) - ^e h{X tm , & m )Eg g(X tm+1 _ k , , £t m+1 -k' ) 

2 q Cr t - k '{\ +r tm ) < 2 q+1 Cr t - k ', 
which clearly satisfies (I2.29p . Since n < u 1 ^, (I2.30P now gives 

/ « \ 2p+2 

E o Q ( W i ~ E °° W i) ^ BnP+1 ^ BuiP+1)/s > 

V j=i J 

and the bound (|2.28j) follows, if we choose s := (p + l)/(p + 2). 

Appendix A. Ergodic lemmas required in the proof 



□ 



The proofs in Section [2] use the ergodic properties of the processes, summarized in the 
following lemmas. 

Lemma A.l. For all integers j > and p > 1, 

Fg {Xj G [6 ,6 + v/n]) < M (A.l) 

and 

%,(l^r + IOI P |^o = ^,eo = y) < rj J R 1 (|^ + |yp)+i2 2 , (A.2) 
u>i£/i a positive constant r\ < 1 and constants R± and R2, independent of Oq. 
Proof. For J > 1, 

G [0 o ,0o + «/n]) =M eo / 
Further, by Jensen's inequality 

fof < (w&-i + (i - I«I)t^I0|) P < HI^-iP + (i - 

and hence 

E eo (l^ p |eo = y)<|a| i |y| p + c 1 . 

Similarly, with p := \p + \ V \p-\ 

\Xtf < (p\X^ x \ + + \e 3 \) P = p\X^ + (j^)' 1 (&-i| p + 



f6o+"/« 2 1 2 f 

'fibiAj G [6^0, 6*0 + -^/^jj = E 9o I e -^-f{Xj-uOo)-Zi-i) dx < 



80 V2vr n 
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E 6o l\Xj\P\X = x,Zo = y) <(J\x\ p + C 2 \a\i\y\ p + Cz, 



which gives (|A.2|) . 



□ 



Lemma A. 2. The Markov chain (Xj,£j) has the unique invariant measure under Fq , 
with uniformly bounded probability density p(x, y; 6q) satisfying 



0o 



[Xj G [6 ,9 + v/n]) 



do+v/n 



00 



p(x,y;6 )dxdy = - / p(e ,y;9o)dy + 0(n 2 ), (A.3) 



where ¥g is the corresponding stationary probability on ($1,9"). 

Moreover, the chain is geometrically ergodic, i.e. there exist positive constants C and 
r < 1, such that for a measurable function \h\ < 1 and m > k 

E eo (h{X m ,U)\X k =x,£ k = y)- E 9o h(X k ^ k )\ < Cr m ~ k (\x\ + \y\), x,y G M, (A.4) 

and consequently, for an J ' m ,oo- measurable random variable \H\ < 1 



E 



9(1 



Ee {H\1 k 



E 9o H 



< Cr 



m—k 



(A.5) 



Finally, (Xj,£j) is geometrically mixing, i.e. for measurable functions \g\ < 1, \h\ < 1 



Ee g(X k ,^ k )h(X k+m , £*:+m) — Eg g(X k , £ k )Eg h(X k+m , 



< Cr m (l + r A 



(A.6) 



In particular, (|A.5P and (|A.6P ZioW £/ie stationary expectation Eg . 

Proof. The transition kernel of the process (Xj,£j) has a positive density with respect to 
the Lebesgue measure: 

1 



2vr 



exp 



■-{u - f(x,6 ) - yf - -(v - ay) 2 ) dudv 



and hence in the terminology of [12J, it is ^-irreducible and aperiodic. Further, a ball 
of radius R > around the origin is a small set with respect to e.g. the measure 

u{dx,dy) := e-^ u2+v2 ^ 1+R ^ u \ + ^dudv 
and V(x,y) = \x\ + \y\ satisfies the drift condition 

(PV)(x,y)-V(x,y) < ~(1 - p A \a\)V(x,y) + 21 {(x>y)eBll} , (x,y)eR 2 , 



for sufficiently large R. By Theorem 15.0.1 [12], it follows that there exists a unique 
invariant probability measure ir and for any measurable h(x,y) < V(a;,y), 



hdir 



<Cr n V(x,y), 



with positive constants C and r < 1, i.e. ()A.4j) holds. Since Eq H = Eg h(X m ,^ m ) and 
E 9o (H\3- k ) = E 9o (h(X m ,U)\? k ) with h(x,y) := E 0o (H\X m = x,U = y), the claim 
follows from (|A.4|) and (IA.2|) . Since the transition kernel P has a bounded continuously 
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differentiable density with respect to the Lebesgue measure, so does the invariant measure 
7r and (1A.3|) follows. The mixing inequality (|A.6j) follows from Theorem 16.1.5 in [12]. □ 

The theory, used in the proof of the previous lemma, does not directly apply to the 
Markov chain (Xj , £j , 3j ) (see (|2.2p for the definition of Sj ) , since it is generated by a 
(3 + (i)-dimensional recursion, driven by two dimensional noise. This typically excludes tp- 
irreducibility. Fortunately, for our purposes the following weaker properties are sufficient: 

Lemma A. 3. The Markov process (Xj,^j,Ej)has the unique invariant measure. Let¥g 
denote the corresponding stationary probability. Then for a measurable function h(x,y,z), 
satisfying \h(x,y,z)\ < 1 and the Lipschitz condition 

\h(x,y,z) - h(x,y,z')\ < L(l + \x\ + \y\ + ||z|| + ||/||)||z - z'\\, x,y £ E, z,z' G R d+1 , 

with a positive constant L, 



Eo (h(X m ,Z m ,Zm)\?e) - E do h(X ^ ,E ) <Cq m ~ l (A.7) 



for some positive constants C and q < 1 and all integers m > £ > 0. 

Proof. Under the stationary measure ¥g from Lemma |A.2[ we can extend the definition 
of (Xj,£j) to the negative integers and define 



-o 



:=c J2 b^Xi-ftXi-^Oo + Uk/n)), k = 0,...,d (A. 



where b := j^— and c := The distribution of (Xo,£o)^o) is invariant. To establish 

uniqueness, let (i and [j! be two invariant measures and note that by Lemma IA.2I their 
(X,£) marginals coincide. Hence 

fj,(dx, dy, dz) = u(dx, dy)/j>(x, y; dz), fi'(dx, dy, dz) = u(dx, dy)fi'(x, y; dz) 

where v is the invariant measure of the process (Xj,£j) and /j,(x, y; dz) and //(x, y; dz) are 
corresponding regular conditional probabilities. Let (Xj,£j,Ej) and (Xj,£j,Ej) be the 
solutions of the recursions (jl.2p . (jl.3p and (jl.7p with u := li^, A; = 0, ...,d subject to the 
initial conditions (Xq,^q,Eq) and (Xq,^q,E' ), where (Xq,^q) is sampled from v and Ho 
and H are sampled from fJ,(Xo, £o! dz) and /jl'(Xq, £o! dz)- Note that E£ — Ej = b J (E' — Eq) 
and hence for any uniformly continuous function g 



gdfj- ~ J gdfjf 



Since uniformly continuous functions form a measure defining class, the uniqueness follows. 



^by uniqueness, the stationary probabilities Pg , introduced in Lemmas IA.2I and IA.3I coincide 
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To derive the bound (|A.7j) . note that: 



z;krrn 



vm-j 



j=e+i 

^ j=e+i 



+ c Yl ( X i ~ f( X J^ d o + u k /n))b m -i =: b^ m ~ e) 4 + J- 
Using the bound (|A.2|) . we get 

E eo |J 1 fe | 2 <2E eo |^| 2 + Ci £ ¥, eo (\X J \ + \X j ^\) 2 \b\^ m+ ^ <C 2 

j=i+i 

By the triangle inequality 

Ee E0 o (h(X m ,£ m , E m )\3e) — Eg^Xo,^,^] 



k 

2 - 



< m. 



(A.9) 



Ea 



< 



E 



0o 



E6» (h(X m , £ m , S m )|9^) — Eg h(X m ,£ m , E 
E4K4,?m^ M Ji + J2pe) ~ Eg (h(X m ,Z m ,J 2 )\?e) 

+ 



+ 



(A.10) 



Eg E eQ {h{X m ^ m , J 2 )\3e) —E 0o h(X m ,£ m ,J 2 ) 

E eo h(X m ,£m, J2) ~ E eo h(X m ,U, b^ m -Vjt + J 2 ) . 
Note that h(X m ,£ m , J 2 ) is measurable with respect to 9"i( m+ ^ ^ and by ()A.5|) 



Ee E eo (/i(X m ,^ m , J 2 )| He) -E eo h(X m ,^ m ,J 2 ) 
By the Lipschitz property of h and (1A.9I) . we have 



E 6o E 6o (h(X m ,S m , b&^Jx + J 2 )\$t) - E flo (/i(X m ,£ m , J 2 )|^ 



< 



E 



h(X m ,U, b^ {m - e) Ji + J 2 ) - h(X m ,U, J2) 
b^ m -^E 6o L(l + \X m \ + \U\ + Pmll + || J 2 ||) || Ji|| < 

(2\ 1/2 
E 0o (l + |X m | + |e m | + ||E m || + ||J 2 ||) J (EeJJiH 2 ) <C 3 6^ m ^. 



Similar bound holds for the last term in (|A.1Q[) and the claim follows with q := y/\b\ V r. 

□ 
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